<!-- HTML header for doxygen 1.8.6-->
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/xhtml;charset=UTF-8"/>
<meta http-equiv="X-UA-Compatible" content="IE=9"/>
<meta name="generator" content="Doxygen 1.8.13"/>
<title>OpenCV: Porting anisotropic image segmentation on G-API</title>
<link href="../../opencv.ico" rel="shortcut icon" type="image/x-icon" />
<link href="../../tabs.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="../../jquery.js"></script>
<script type="text/javascript" src="../../dynsections.js"></script>
<script type="text/javascript" src="../../tutorial-utils.js"></script>
<link href="../../search/search.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="../../search/searchdata.js"></script>
<script type="text/javascript" src="../../search/search.js"></script>
<script type="text/x-mathjax-config">
  MathJax.Hub.Config({
    extensions: ["tex2jax.js", "TeX/AMSmath.js", "TeX/AMSsymbols.js"],
    jax: ["input/TeX","output/HTML-CSS"],
});
//<![CDATA[
MathJax.Hub.Config(
{
  TeX: {
      Macros: {
          matTT: [ "\\[ \\left|\\begin{array}{ccc} #1 & #2 & #3\\\\ #4 & #5 & #6\\\\ #7 & #8 & #9 \\end{array}\\right| \\]", 9],
          fork: ["\\left\\{ \\begin{array}{l l} #1 & \\mbox{#2}\\\\ #3 & \\mbox{#4}\\\\ \\end{array} \\right.", 4],
          forkthree: ["\\left\\{ \\begin{array}{l l} #1 & \\mbox{#2}\\\\ #3 & \\mbox{#4}\\\\ #5 & \\mbox{#6}\\\\ \\end{array} \\right.", 6],
          forkfour: ["\\left\\{ \\begin{array}{l l} #1 & \\mbox{#2}\\\\ #3 & \\mbox{#4}\\\\ #5 & \\mbox{#6}\\\\ #7 & \\mbox{#8}\\\\ \\end{array} \\right.", 8],
          vecthree: ["\\begin{bmatrix} #1\\\\ #2\\\\ #3 \\end{bmatrix}", 3],
          vecthreethree: ["\\begin{bmatrix} #1 & #2 & #3\\\\ #4 & #5 & #6\\\\ #7 & #8 & #9 \\end{bmatrix}", 9],
          cameramatrix: ["#1 = \\begin{bmatrix} f_x & 0 & c_x\\\\ 0 & f_y & c_y\\\\ 0 & 0 & 1 \\end{bmatrix}", 1],
          distcoeffs: ["(k_1, k_2, p_1, p_2[, k_3[, k_4, k_5, k_6 [, s_1, s_2, s_3, s_4[, \\tau_x, \\tau_y]]]]) \\text{ of 4, 5, 8, 12 or 14 elements}"],
          distcoeffsfisheye: ["(k_1, k_2, k_3, k_4)"],
          hdotsfor: ["\\dots", 1],
          mathbbm: ["\\mathbb{#1}", 1],
          bordermatrix: ["\\matrix{#1}", 1]
      }
  }
}
);
//]]>
</script><script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js"></script>
<link href="../../doxygen.css" rel="stylesheet" type="text/css" />
<link href="../../stylesheet.css" rel="stylesheet" type="text/css"/>
</head>
<body>
<div id="top"><!-- do not remove this div, it is closed by doxygen! -->
<div id="titlearea">
<!--#include virtual="/google-search.html"-->
<table cellspacing="0" cellpadding="0">
 <tbody>
 <tr style="height: 56px;">
  <td id="projectlogo"><img alt="Logo" src="../../opencv-logo-small.png"/></td>
  <td style="padding-left: 0.5em;">
   <div id="projectname">OpenCV
   &#160;<span id="projectnumber">4.5.2</span>
   </div>
   <div id="projectbrief">Open Source Computer Vision</div>
  </td>
 </tr>
 </tbody>
</table>
</div>
<!-- end header part -->
<!-- Generated by Doxygen 1.8.13 -->
<script type="text/javascript">
var searchBox = new SearchBox("searchBox", "../../search",false,'Search');
</script>
<script type="text/javascript" src="../../menudata.js"></script>
<script type="text/javascript" src="../../menu.js"></script>
<script type="text/javascript">
$(function() {
  initMenu('../../',true,false,'search.php','Search');
  $(document).ready(function() { init_search(); });
});
</script>
<div id="main-nav"></div>
<!-- window showing the filter options -->
<div id="MSearchSelectWindow"
     onmouseover="return searchBox.OnSearchSelectShow()"
     onmouseout="return searchBox.OnSearchSelectHide()"
     onkeydown="return searchBox.OnSearchSelectKey(event)">
</div>

<!-- iframe showing the search results (closed by default) -->
<div id="MSearchResultsWindow">
<iframe src="javascript:void(0)" frameborder="0" 
        name="MSearchResults" id="MSearchResults">
</iframe>
</div>

<div id="nav-path" class="navpath">
  <ul>
<li class="navelem"><a class="el" href="../../d9/df8/tutorial_root.html">OpenCV Tutorials</a></li><li class="navelem"><a class="el" href="../../df/d7e/tutorial_table_of_content_gapi.html">Graph API (gapi module)</a></li>  </ul>
</div>
</div><!-- top -->
<div class="header">
  <div class="headertitle">
<div class="title">Porting anisotropic image segmentation on G-API </div>  </div>
</div><!--header-->
<div class="contents">
<div class="toc"><h3>Table of Contents</h3>
<ul><li class="level1"><a href="#gapi_anisotropic_intro">Introduction</a></li>
<li class="level1"><a href="#gapi_anisotropic_start">Quick start: using OpenCV backend</a><ul><li class="level2"><a href="#gapi_anisotropic_calcgst">Examining calcGST()</a></li>
<li class="level2"><a href="#gapi_anisotropic_running">Running G-API graph</a></li>
<li class="level2"><a href="#gapi_anisotropic_ocv">G-API initial version: full listing</a></li>
</ul>
</li>
<li class="level1"><a href="#gapi_anisotropic_inspect">Inspecting the initial version</a><ul><li class="level2"><a href="#gapi_anisotropic_inspect_graph">Understanding the graph structure</a></li>
<li class="level2"><a href="#gapi_anisotropic_memory_ocv">Measuring memory footprint</a></li>
</ul>
</li>
<li class="level1"><a href="#gapi_anisotropic_backends">Backends and kernels</a><ul><li class="level2"><a href="#gapi_anisotropic_fluid">Running a graph with a Fluid backend</a></li>
<li class="level2"><a href="#gapi_anisotropic_trouble">Troubleshooting and customization</a></li>
</ul>
</li>
<li class="level1"><a href="#gapi_tutor_conclusion">Conclusion</a></li>
</ul>
</div>
<div class="textblock"><p><b>Prev Tutorial:</b> <a class="el" href="../../d8/d24/tutorial_gapi_interactive_face_detection.html">Face analytics pipeline with G-API</a></p>
<p><b>Next Tutorial:</b> <a class="el" href="../../d4/d48/tutorial_gapi_face_beautification.html">Implementing a face beautification algorithm with G-API</a></p>
<h1><a class="anchor" id="gapi_anisotropic_intro"></a>
Introduction</h1>
<p>In this tutorial you will learn:</p><ul>
<li>How an existing algorithm can be transformed into a G-API computation (graph);</li>
<li>How to inspect and profile G-API graphs;</li>
<li>How to customize graph execution without changing its code.</li>
</ul>
<p>This tutorial is based on <a class="el" href="../../d4/d70/tutorial_anisotropic_image_segmentation_by_a_gst.html">Anisotropic image segmentation by a gradient structure tensor</a>.</p>
<h1><a class="anchor" id="gapi_anisotropic_start"></a>
Quick start: using OpenCV backend</h1>
<p>Before we start, let's review the original algorithm implementation:</p>
<div class="fragment"><div class="line"></div><div class="line"><span class="preprocessor">#include &lt;iostream&gt;</span></div><div class="line"><span class="preprocessor">#include &quot;<a class="code" href="../../d1/d4f/imgproc_2include_2opencv2_2imgproc_8hpp.html">opencv2/imgproc.hpp</a>&quot;</span></div><div class="line"><span class="preprocessor">#include &quot;<a class="code" href="../../d6/d87/imgcodecs_8hpp.html">opencv2/imgcodecs.hpp</a>&quot;</span></div><div class="line"></div><div class="line"><span class="keyword">using namespace </span><a class="code" href="../../d2/d75/namespacecv.html">cv</a>;</div><div class="line"><span class="keyword">using namespace </span>std;</div><div class="line"></div><div class="line"><span class="keywordtype">void</span> calcGST(<span class="keyword">const</span> <a class="code" href="../../d3/d63/classcv_1_1Mat.html">Mat</a>&amp; inputImg, <a class="code" href="../../d3/d63/classcv_1_1Mat.html">Mat</a>&amp; imgCoherencyOut, <a class="code" href="../../d3/d63/classcv_1_1Mat.html">Mat</a>&amp; imgOrientationOut, <span class="keywordtype">int</span> w);</div><div class="line"></div><div class="line"><span class="keywordtype">int</span> main()</div><div class="line">{</div><div class="line">    <span class="keywordtype">int</span> W = 52;             <span class="comment">// window size is WxW</span></div><div class="line">    <span class="keywordtype">double</span> C_Thr = 0.43;    <span class="comment">// threshold for coherency</span></div><div class="line">    <span class="keywordtype">int</span> LowThr = 35;        <span class="comment">// threshold1 for orientation, it ranges from 0 to 180</span></div><div class="line">    <span class="keywordtype">int</span> HighThr = 57;       <span class="comment">// threshold2 for orientation, it ranges from 0 to 180</span></div><div class="line"></div><div class="line">    <a class="code" href="../../d3/d63/classcv_1_1Mat.html">Mat</a> imgIn = <a class="code" href="../../d4/da8/group__imgcodecs.html#ga288b8b3da0892bd651fce07b3bbd3a56">imread</a>(<span class="stringliteral">&quot;input.jpg&quot;</span>, <a class="code" href="../../d8/d6a/group__imgcodecs__flags.html#gga61d9b0126a3e57d9277ac48327799c80ae29981cfc153d3b0cef5c0daeedd2125">IMREAD_GRAYSCALE</a>);</div><div class="line">    <span class="keywordflow">if</span> (imgIn.<a class="code" href="../../d3/d63/classcv_1_1Mat.html#abbec3525a852e77998aba034813fded4">empty</a>()) <span class="comment">//check whether the image is loaded or not</span></div><div class="line">    {</div><div class="line">        cout &lt;&lt; <span class="stringliteral">&quot;ERROR : Image cannot be loaded..!!&quot;</span> &lt;&lt; endl;</div><div class="line">        <span class="keywordflow">return</span> -1;</div><div class="line">    }</div><div class="line"></div><div class="line">    <a class="code" href="../../d3/d63/classcv_1_1Mat.html">Mat</a> imgCoherency, imgOrientation;</div><div class="line">    calcGST(imgIn, imgCoherency, imgOrientation, W);</div><div class="line"></div><div class="line">    <a class="code" href="../../d3/d63/classcv_1_1Mat.html">Mat</a> imgCoherencyBin;</div><div class="line">    imgCoherencyBin = imgCoherency &gt; C_Thr;</div><div class="line">    <a class="code" href="../../d3/d63/classcv_1_1Mat.html">Mat</a> imgOrientationBin;</div><div class="line">    <a class="code" href="../../d2/de8/group__core__array.html#ga48af0ab51e36436c5d04340e036ce981">inRange</a>(imgOrientation, <a class="code" href="../../dc/d84/group__core__basic.html#ga599fe92e910c027be274233eccad7beb">Scalar</a>(LowThr), <a class="code" href="../../dc/d84/group__core__basic.html#ga599fe92e910c027be274233eccad7beb">Scalar</a>(HighThr), imgOrientationBin);</div><div class="line"></div><div class="line">    <a class="code" href="../../d3/d63/classcv_1_1Mat.html">Mat</a> imgBin;</div><div class="line">    imgBin = imgCoherencyBin &amp; imgOrientationBin;</div><div class="line"></div><div class="line">    <a class="code" href="../../dc/d84/group__core__basic.html#ga1b6a396a456c8b6c6e4afd8591560d80">normalize</a>(imgCoherency, imgCoherency, 0, 255, <a class="code" href="../../d2/de8/group__core__array.html#ggad12cefbcb5291cf958a85b4b67b6149fa9f0c1c342a18114d47b516a88e29822e">NORM_MINMAX</a>);</div><div class="line">    <a class="code" href="../../dc/d84/group__core__basic.html#ga1b6a396a456c8b6c6e4afd8591560d80">normalize</a>(imgOrientation, imgOrientation, 0, 255, <a class="code" href="../../d2/de8/group__core__array.html#ggad12cefbcb5291cf958a85b4b67b6149fa9f0c1c342a18114d47b516a88e29822e">NORM_MINMAX</a>);</div><div class="line"></div><div class="line">    <a class="code" href="../../d4/da8/group__imgcodecs.html#gabbc7ef1aa2edfaa87772f1202d67e0ce">imwrite</a>(<span class="stringliteral">&quot;result.jpg&quot;</span>, 0.5*(imgIn + imgBin));</div><div class="line">    <a class="code" href="../../d4/da8/group__imgcodecs.html#gabbc7ef1aa2edfaa87772f1202d67e0ce">imwrite</a>(<span class="stringliteral">&quot;Coherency.jpg&quot;</span>, imgCoherency);</div><div class="line">    <a class="code" href="../../d4/da8/group__imgcodecs.html#gabbc7ef1aa2edfaa87772f1202d67e0ce">imwrite</a>(<span class="stringliteral">&quot;Orientation.jpg&quot;</span>, imgOrientation);</div><div class="line">    <span class="keywordflow">return</span> 0;</div><div class="line">}</div><div class="line"><span class="keywordtype">void</span> calcGST(<span class="keyword">const</span> <a class="code" href="../../d3/d63/classcv_1_1Mat.html">Mat</a>&amp; inputImg, <a class="code" href="../../d3/d63/classcv_1_1Mat.html">Mat</a>&amp; imgCoherencyOut, <a class="code" href="../../d3/d63/classcv_1_1Mat.html">Mat</a>&amp; imgOrientationOut, <span class="keywordtype">int</span> w)</div><div class="line">{</div><div class="line">    <a class="code" href="../../d3/d63/classcv_1_1Mat.html">Mat</a> img;</div><div class="line">    inputImg.<a class="code" href="../../d3/d63/classcv_1_1Mat.html#adf88c60c5b4980e05bb556080916978b">convertTo</a>(img, <a class="code" href="../../d1/d1b/group__core__hal__interface.html#ga4a3def5d72b74bed31f5f8ab7676099c">CV_32F</a>);</div><div class="line"></div><div class="line">    <span class="comment">// GST components calculation (start)</span></div><div class="line">    <span class="comment">// J =  (J11 J12; J12 J22) - GST</span></div><div class="line">    <a class="code" href="../../d3/d63/classcv_1_1Mat.html">Mat</a> imgDiffX, imgDiffY, imgDiffXY;</div><div class="line">    <a class="code" href="../../d4/d86/group__imgproc__filter.html#gacea54f142e81b6758cb6f375ce782c8d">Sobel</a>(img, imgDiffX, <a class="code" href="../../d1/d1b/group__core__hal__interface.html#ga4a3def5d72b74bed31f5f8ab7676099c">CV_32F</a>, 1, 0, 3);</div><div class="line">    <a class="code" href="../../d4/d86/group__imgproc__filter.html#gacea54f142e81b6758cb6f375ce782c8d">Sobel</a>(img, imgDiffY, <a class="code" href="../../d1/d1b/group__core__hal__interface.html#ga4a3def5d72b74bed31f5f8ab7676099c">CV_32F</a>, 0, 1, 3);</div><div class="line">    <a class="code" href="../../d2/de8/group__core__array.html#ga979d898a58d7f61c53003e162e7ad89f">multiply</a>(imgDiffX, imgDiffY, imgDiffXY);</div><div class="line"></div><div class="line">    <a class="code" href="../../d3/d63/classcv_1_1Mat.html">Mat</a> imgDiffXX, imgDiffYY;</div><div class="line">    <a class="code" href="../../d2/de8/group__core__array.html#ga979d898a58d7f61c53003e162e7ad89f">multiply</a>(imgDiffX, imgDiffX, imgDiffXX);</div><div class="line">    <a class="code" href="../../d2/de8/group__core__array.html#ga979d898a58d7f61c53003e162e7ad89f">multiply</a>(imgDiffY, imgDiffY, imgDiffYY);</div><div class="line"></div><div class="line">    <a class="code" href="../../d3/d63/classcv_1_1Mat.html">Mat</a> J11, J22, J12;      <span class="comment">// J11, J22 and J12 are GST components</span></div><div class="line">    <a class="code" href="../../d4/d86/group__imgproc__filter.html#gad533230ebf2d42509547d514f7d3fbc3">boxFilter</a>(imgDiffXX, J11, <a class="code" href="../../d1/d1b/group__core__hal__interface.html#ga4a3def5d72b74bed31f5f8ab7676099c">CV_32F</a>, <a class="code" href="../../dc/d84/group__core__basic.html#ga346f563897249351a34549137c8532a0">Size</a>(w, w));</div><div class="line">    <a class="code" href="../../d4/d86/group__imgproc__filter.html#gad533230ebf2d42509547d514f7d3fbc3">boxFilter</a>(imgDiffYY, J22, <a class="code" href="../../d1/d1b/group__core__hal__interface.html#ga4a3def5d72b74bed31f5f8ab7676099c">CV_32F</a>, <a class="code" href="../../dc/d84/group__core__basic.html#ga346f563897249351a34549137c8532a0">Size</a>(w, w));</div><div class="line">    <a class="code" href="../../d4/d86/group__imgproc__filter.html#gad533230ebf2d42509547d514f7d3fbc3">boxFilter</a>(imgDiffXY, J12, <a class="code" href="../../d1/d1b/group__core__hal__interface.html#ga4a3def5d72b74bed31f5f8ab7676099c">CV_32F</a>, <a class="code" href="../../dc/d84/group__core__basic.html#ga346f563897249351a34549137c8532a0">Size</a>(w, w));</div><div class="line">    <span class="comment">// GST components calculation (stop)</span></div><div class="line"></div><div class="line">    <span class="comment">// eigenvalue calculation (start)</span></div><div class="line">    <span class="comment">// lambda1 = 0.5*(J11 + J22 + sqrt((J11-J22)^2 + 4*J12^2))</span></div><div class="line">    <span class="comment">// lambda2 = 0.5*(J11 + J22 - sqrt((J11-J22)^2 + 4*J12^2))</span></div><div class="line">    <a class="code" href="../../d3/d63/classcv_1_1Mat.html">Mat</a> tmp1, tmp2, tmp3, tmp4;</div><div class="line">    tmp1 = J11 + J22;</div><div class="line">    tmp2 = J11 - J22;</div><div class="line">    <a class="code" href="../../d2/de8/group__core__array.html#ga979d898a58d7f61c53003e162e7ad89f">multiply</a>(tmp2, tmp2, tmp2);</div><div class="line">    <a class="code" href="../../d2/de8/group__core__array.html#ga979d898a58d7f61c53003e162e7ad89f">multiply</a>(J12, J12, tmp3);</div><div class="line">    <a class="code" href="../../d0/de1/group__core.html#ga9070b6a3f093dd952d973819b06f4906">sqrt</a>(tmp2 + 4.0 * tmp3, tmp4);</div><div class="line"></div><div class="line">    <a class="code" href="../../d3/d63/classcv_1_1Mat.html">Mat</a> lambda1, lambda2;</div><div class="line">    lambda1 = tmp1 + tmp4;</div><div class="line">    lambda1 = 0.5*lambda1;      <span class="comment">// biggest eigenvalue</span></div><div class="line">    lambda2 = tmp1 - tmp4;</div><div class="line">    lambda2 = 0.5*lambda2;      <span class="comment">// smallest eigenvalue</span></div><div class="line">    <span class="comment">// eigenvalue calculation (stop)</span></div><div class="line"></div><div class="line">    <span class="comment">// Coherency calculation (start)</span></div><div class="line">    <span class="comment">// Coherency = (lambda1 - lambda2)/(lambda1 + lambda2)) - measure of anisotropism</span></div><div class="line">    <span class="comment">// Coherency is anisotropy degree (consistency of local orientation)</span></div><div class="line">    <a class="code" href="../../d2/de8/group__core__array.html#ga6db555d30115642fedae0cda05604874">divide</a>(lambda1 - lambda2, lambda1 + lambda2, imgCoherencyOut);</div><div class="line">    <span class="comment">// Coherency calculation (stop)</span></div><div class="line"></div><div class="line">    <span class="comment">// orientation angle calculation (start)</span></div><div class="line">    <span class="comment">// tan(2*Alpha) = 2*J12/(J22 - J11)</span></div><div class="line">    <span class="comment">// Alpha = 0.5 atan2(2*J12/(J22 - J11))</span></div><div class="line">    <a class="code" href="../../d2/de8/group__core__array.html#ga9db9ca9b4d81c3bde5677b8f64dc0137">phase</a>(J22 - J11, 2.0*J12, imgOrientationOut, <span class="keyword">true</span>);</div><div class="line">    imgOrientationOut = 0.5*imgOrientationOut;</div><div class="line">    <span class="comment">// orientation angle calculation (stop)</span></div><div class="line">}</div></div><!-- fragment --><h2><a class="anchor" id="gapi_anisotropic_calcgst"></a>
Examining calcGST()</h2>
<p>The function calcGST() is clearly an image processing pipeline:</p><ul>
<li>It is just a sequence of operations over a number of <a class="el" href="../../d3/d63/classcv_1_1Mat.html" title="n-dimensional dense array class ">cv::Mat</a>;</li>
<li>No logic (conditionals) and loops involved in the code;</li>
<li>All functions operate on 2D images (like <a class="el" href="../../d4/d86/group__imgproc__filter.html#gacea54f142e81b6758cb6f375ce782c8d" title="Calculates the first, second, third, or mixed image derivatives using an extended Sobel operator...">cv::Sobel</a>, <a class="el" href="../../d2/de8/group__core__array.html#ga979d898a58d7f61c53003e162e7ad89f" title="Calculates the per-element scaled product of two arrays. ">cv::multiply</a>, <a class="el" href="../../d4/d86/group__imgproc__filter.html#gad533230ebf2d42509547d514f7d3fbc3" title="Blurs an image using the box filter. ">cv::boxFilter</a>, <a class="el" href="../../d0/de1/group__core.html#ga9070b6a3f093dd952d973819b06f4906">cv::sqrt</a>, etc).</li>
</ul>
<p>Considering the above, calcGST() is a great candidate to start with. In the original code, its prototype is defined like this:</p>
<div class="fragment"><div class="line"><span class="keywordtype">void</span> calcGST(<span class="keyword">const</span> Mat&amp; inputImg, Mat&amp; imgCoherencyOut, Mat&amp; imgOrientationOut, <span class="keywordtype">int</span> w);</div></div><!-- fragment --><p> With G-API, we can define it as follows:</p>
<div class="fragment"><div class="line"><span class="keywordtype">void</span> calcGST(<span class="keyword">const</span> <a class="code" href="../../df/daa/classcv_1_1GMat.html">cv::GMat</a>&amp; inputImg, <a class="code" href="../../df/daa/classcv_1_1GMat.html">cv::GMat</a>&amp; imgCoherencyOut, <a class="code" href="../../df/daa/classcv_1_1GMat.html">cv::GMat</a>&amp; imgOrientationOut, <span class="keywordtype">int</span> w);</div></div><!-- fragment --><p> It is important to understand that the new G-API based version of calcGST() will just produce a compute graph, in contrast to its original version, which actually calculates the values. This is a principal difference &ndash; G-API based functions like this are used to construct graphs, not to process the actual data.</p>
<p>Let's start implementing calcGST() with calculation of \(J\) matrix. This is how the original code looks like:</p>
<div class="fragment"><div class="line"><span class="keywordtype">void</span> calcGST(<span class="keyword">const</span> Mat&amp; inputImg, Mat&amp; imgCoherencyOut, Mat&amp; imgOrientationOut, <span class="keywordtype">int</span> w)</div><div class="line">{</div><div class="line">    Mat img;</div><div class="line">    inputImg.<a class="code" href="../../d3/d63/classcv_1_1Mat.html#adf88c60c5b4980e05bb556080916978b">convertTo</a>(img, <a class="code" href="../../d1/d1b/group__core__hal__interface.html#ga4a3def5d72b74bed31f5f8ab7676099c">CV_32F</a>);</div><div class="line"></div><div class="line">    <span class="comment">// GST components calculation (start)</span></div><div class="line">    <span class="comment">// J =  (J11 J12; J12 J22) - GST</span></div><div class="line">    Mat imgDiffX, imgDiffY, imgDiffXY;</div><div class="line">    <a class="code" href="../../d4/d86/group__imgproc__filter.html#gacea54f142e81b6758cb6f375ce782c8d">Sobel</a>(img, imgDiffX, <a class="code" href="../../d1/d1b/group__core__hal__interface.html#ga4a3def5d72b74bed31f5f8ab7676099c">CV_32F</a>, 1, 0, 3);</div><div class="line">    <a class="code" href="../../d4/d86/group__imgproc__filter.html#gacea54f142e81b6758cb6f375ce782c8d">Sobel</a>(img, imgDiffY, <a class="code" href="../../d1/d1b/group__core__hal__interface.html#ga4a3def5d72b74bed31f5f8ab7676099c">CV_32F</a>, 0, 1, 3);</div><div class="line">    <a class="code" href="../../d2/de8/group__core__array.html#ga979d898a58d7f61c53003e162e7ad89f">multiply</a>(imgDiffX, imgDiffY, imgDiffXY);</div></div><!-- fragment --><p> Here we need to declare output objects for every new operation (see img as a result for <a class="el" href="../../d3/d63/classcv_1_1Mat.html#adf88c60c5b4980e05bb556080916978b" title="Converts an array to another data type with optional scaling. ">cv::Mat::convertTo</a>, imgDiffX and others as results for <a class="el" href="../../d4/d86/group__imgproc__filter.html#gacea54f142e81b6758cb6f375ce782c8d" title="Calculates the first, second, third, or mixed image derivatives using an extended Sobel operator...">cv::Sobel</a> and <a class="el" href="../../d2/de8/group__core__array.html#ga979d898a58d7f61c53003e162e7ad89f" title="Calculates the per-element scaled product of two arrays. ">cv::multiply</a>).</p>
<p>The G-API analogue is listed below:</p>
<div class="fragment"><div class="line"><span class="keywordtype">void</span> calcGST(<span class="keyword">const</span> <a class="code" href="../../df/daa/classcv_1_1GMat.html">cv::GMat</a>&amp; inputImg, <a class="code" href="../../df/daa/classcv_1_1GMat.html">cv::GMat</a>&amp; imgCoherencyOut, <a class="code" href="../../df/daa/classcv_1_1GMat.html">cv::GMat</a>&amp; imgOrientationOut, <span class="keywordtype">int</span> w)</div><div class="line">{</div><div class="line">    <span class="keyword">auto</span> img = <a class="code" href="../../d6/d91/group__gapi__transform.html#gae972b2c91904d36e92ee14a6778db82e">cv::gapi::convertTo</a>(inputImg, <a class="code" href="../../d1/d1b/group__core__hal__interface.html#ga4a3def5d72b74bed31f5f8ab7676099c">CV_32F</a>);</div><div class="line">    <span class="keyword">auto</span> imgDiffX = <a class="code" href="../../da/dc5/group__gapi__filters.html#gae1443445c4cb3187dcf439a57cfa534f">cv::gapi::Sobel</a>(img, <a class="code" href="../../d1/d1b/group__core__hal__interface.html#ga4a3def5d72b74bed31f5f8ab7676099c">CV_32F</a>, 1, 0, 3);</div><div class="line">    <span class="keyword">auto</span> imgDiffY = <a class="code" href="../../da/dc5/group__gapi__filters.html#gae1443445c4cb3187dcf439a57cfa534f">cv::gapi::Sobel</a>(img, <a class="code" href="../../d1/d1b/group__core__hal__interface.html#ga4a3def5d72b74bed31f5f8ab7676099c">CV_32F</a>, 0, 1, 3);</div><div class="line">    <span class="keyword">auto</span> imgDiffXY = <a class="code" href="../../da/dd3/group__gapi__math.html#ga42a8187b2bf350f90e4a31f93e929e51">cv::gapi::mul</a>(imgDiffX, imgDiffY);</div></div><!-- fragment --><p> This snippet demonstrates the following syntactic difference between G-API and traditional OpenCV:</p><ul>
<li>All standard G-API functions are by default placed in "cv::gapi" namespace;</li>
<li>G-API operations <em>return</em> its results &ndash; there's no need to pass extra "output" parameters to the functions.</li>
</ul>
<p>Note &ndash; this code is also using <code>auto</code> &ndash; types of intermediate objects like <code>img</code>, <code>imgDiffX</code>, and so on are inferred automatically by the C++ compiler. In this example, the types are determined by G-API operation return values which all are <a class="el" href="../../df/daa/classcv_1_1GMat.html">cv::GMat</a>.</p>
<p>G-API standard kernels are trying to follow OpenCV API conventions whenever possible &ndash; so cv::gapi::sobel takes the same arguments as <a class="el" href="../../d4/d86/group__imgproc__filter.html#gacea54f142e81b6758cb6f375ce782c8d" title="Calculates the first, second, third, or mixed image derivatives using an extended Sobel operator...">cv::Sobel</a>, <a class="el" href="../../da/dd3/group__gapi__math.html#ga42a8187b2bf350f90e4a31f93e929e51" title="Calculates the per-element scaled product of two matrices. ">cv::gapi::mul</a> follows <a class="el" href="../../d2/de8/group__core__array.html#ga979d898a58d7f61c53003e162e7ad89f" title="Calculates the per-element scaled product of two arrays. ">cv::multiply</a>, and so on (except having a return value).</p>
<p>The rest of calcGST() function can be implemented the same way trivially. Below is its full source code:</p>
<div class="fragment"><div class="line"><span class="keywordtype">void</span> calcGST(<span class="keyword">const</span> <a class="code" href="../../df/daa/classcv_1_1GMat.html">cv::GMat</a>&amp; inputImg, <a class="code" href="../../df/daa/classcv_1_1GMat.html">cv::GMat</a>&amp; imgCoherencyOut, <a class="code" href="../../df/daa/classcv_1_1GMat.html">cv::GMat</a>&amp; imgOrientationOut, <span class="keywordtype">int</span> w)</div><div class="line">{</div><div class="line">    <span class="keyword">auto</span> img = <a class="code" href="../../d6/d91/group__gapi__transform.html#gae972b2c91904d36e92ee14a6778db82e">cv::gapi::convertTo</a>(inputImg, <a class="code" href="../../d1/d1b/group__core__hal__interface.html#ga4a3def5d72b74bed31f5f8ab7676099c">CV_32F</a>);</div><div class="line">    <span class="keyword">auto</span> imgDiffX = <a class="code" href="../../da/dc5/group__gapi__filters.html#gae1443445c4cb3187dcf439a57cfa534f">cv::gapi::Sobel</a>(img, <a class="code" href="../../d1/d1b/group__core__hal__interface.html#ga4a3def5d72b74bed31f5f8ab7676099c">CV_32F</a>, 1, 0, 3);</div><div class="line">    <span class="keyword">auto</span> imgDiffY = <a class="code" href="../../da/dc5/group__gapi__filters.html#gae1443445c4cb3187dcf439a57cfa534f">cv::gapi::Sobel</a>(img, <a class="code" href="../../d1/d1b/group__core__hal__interface.html#ga4a3def5d72b74bed31f5f8ab7676099c">CV_32F</a>, 0, 1, 3);</div><div class="line">    <span class="keyword">auto</span> imgDiffXY = <a class="code" href="../../da/dd3/group__gapi__math.html#ga42a8187b2bf350f90e4a31f93e929e51">cv::gapi::mul</a>(imgDiffX, imgDiffY);</div><div class="line"></div><div class="line">    <span class="keyword">auto</span> imgDiffXX = <a class="code" href="../../da/dd3/group__gapi__math.html#ga42a8187b2bf350f90e4a31f93e929e51">cv::gapi::mul</a>(imgDiffX, imgDiffX);</div><div class="line">    <span class="keyword">auto</span> imgDiffYY = <a class="code" href="../../da/dd3/group__gapi__math.html#ga42a8187b2bf350f90e4a31f93e929e51">cv::gapi::mul</a>(imgDiffY, imgDiffY);</div><div class="line"></div><div class="line">    <span class="keyword">auto</span> J11 = <a class="code" href="../../da/dc5/group__gapi__filters.html#gaa37a58b30753913cbcc7f5425726dbd5">cv::gapi::boxFilter</a>(imgDiffXX, <a class="code" href="../../d1/d1b/group__core__hal__interface.html#ga4a3def5d72b74bed31f5f8ab7676099c">CV_32F</a>, <a class="code" href="../../d6/d50/classcv_1_1Size__.html">cv::Size</a>(w, w));</div><div class="line">    <span class="keyword">auto</span> J22 = <a class="code" href="../../da/dc5/group__gapi__filters.html#gaa37a58b30753913cbcc7f5425726dbd5">cv::gapi::boxFilter</a>(imgDiffYY, <a class="code" href="../../d1/d1b/group__core__hal__interface.html#ga4a3def5d72b74bed31f5f8ab7676099c">CV_32F</a>, <a class="code" href="../../d6/d50/classcv_1_1Size__.html">cv::Size</a>(w, w));</div><div class="line">    <span class="keyword">auto</span> J12 = <a class="code" href="../../da/dc5/group__gapi__filters.html#gaa37a58b30753913cbcc7f5425726dbd5">cv::gapi::boxFilter</a>(imgDiffXY, <a class="code" href="../../d1/d1b/group__core__hal__interface.html#ga4a3def5d72b74bed31f5f8ab7676099c">CV_32F</a>, <a class="code" href="../../d6/d50/classcv_1_1Size__.html">cv::Size</a>(w, w));</div><div class="line"></div><div class="line">    <span class="keyword">auto</span> tmp1 = J11 + J22;</div><div class="line">    <span class="keyword">auto</span> tmp2 = J11 - J22;</div><div class="line">    <span class="keyword">auto</span> tmp22 = <a class="code" href="../../da/dd3/group__gapi__math.html#ga42a8187b2bf350f90e4a31f93e929e51">cv::gapi::mul</a>(tmp2, tmp2);</div><div class="line">    <span class="keyword">auto</span> tmp3 = <a class="code" href="../../da/dd3/group__gapi__math.html#ga42a8187b2bf350f90e4a31f93e929e51">cv::gapi::mul</a>(J12, J12);</div><div class="line">    <span class="keyword">auto</span> tmp4 = <a class="code" href="../../da/dd3/group__gapi__math.html#gaf3880c8feae9027fa91a701a1fd0d3b4">cv::gapi::sqrt</a>(tmp22 + 4.0*tmp3);</div><div class="line"></div><div class="line">    <span class="keyword">auto</span> lambda1 = tmp1 + tmp4;</div><div class="line">    <span class="keyword">auto</span> lambda2 = tmp1 - tmp4;</div><div class="line"></div><div class="line">    imgCoherencyOut = (lambda1 - lambda2) / (lambda1 + lambda2);</div><div class="line">    imgOrientationOut = 0.5*<a class="code" href="../../da/dd3/group__gapi__math.html#ga3c98a8ad1054d296500220737c04d18c">cv::gapi::phase</a>(J22 - J11, 2.0*J12, <span class="keyword">true</span>);</div><div class="line">}</div></div><!-- fragment --> <h2><a class="anchor" id="gapi_anisotropic_running"></a>
Running G-API graph</h2>
<p>After calcGST() is defined in G-API language, we can construct a graph based on it and finally run it &ndash; pass input image and obtain result. Before we do it, let's have a look how original code looked like:</p>
<div class="fragment"><div class="line">    Mat imgCoherency, imgOrientation;</div><div class="line">    calcGST(imgIn, imgCoherency, imgOrientation, W);</div><div class="line"></div><div class="line">    Mat imgCoherencyBin;</div><div class="line">    imgCoherencyBin = imgCoherency &gt; C_Thr;</div><div class="line">    Mat imgOrientationBin;</div><div class="line">    <a class="code" href="../../d2/de8/group__core__array.html#ga48af0ab51e36436c5d04340e036ce981">inRange</a>(imgOrientation, <a class="code" href="../../dc/d84/group__core__basic.html#ga599fe92e910c027be274233eccad7beb">Scalar</a>(LowThr), <a class="code" href="../../dc/d84/group__core__basic.html#ga599fe92e910c027be274233eccad7beb">Scalar</a>(HighThr), imgOrientationBin);</div><div class="line"></div><div class="line">    Mat imgBin;</div><div class="line">    imgBin = imgCoherencyBin &amp; imgOrientationBin;</div><div class="line"></div><div class="line">    <a class="code" href="../../dc/d84/group__core__basic.html#ga1b6a396a456c8b6c6e4afd8591560d80">normalize</a>(imgCoherency, imgCoherency, 0, 255, <a class="code" href="../../d2/de8/group__core__array.html#ggad12cefbcb5291cf958a85b4b67b6149fa9f0c1c342a18114d47b516a88e29822e">NORM_MINMAX</a>);</div><div class="line">    <a class="code" href="../../dc/d84/group__core__basic.html#ga1b6a396a456c8b6c6e4afd8591560d80">normalize</a>(imgOrientation, imgOrientation, 0, 255, <a class="code" href="../../d2/de8/group__core__array.html#ggad12cefbcb5291cf958a85b4b67b6149fa9f0c1c342a18114d47b516a88e29822e">NORM_MINMAX</a>);</div><div class="line"></div><div class="line">    <a class="code" href="../../d4/da8/group__imgcodecs.html#gabbc7ef1aa2edfaa87772f1202d67e0ce">imwrite</a>(<span class="stringliteral">&quot;result.jpg&quot;</span>, 0.5*(imgIn + imgBin));</div><div class="line">    <a class="code" href="../../d4/da8/group__imgcodecs.html#gabbc7ef1aa2edfaa87772f1202d67e0ce">imwrite</a>(<span class="stringliteral">&quot;Coherency.jpg&quot;</span>, imgCoherency);</div><div class="line">    <a class="code" href="../../d4/da8/group__imgcodecs.html#gabbc7ef1aa2edfaa87772f1202d67e0ce">imwrite</a>(<span class="stringliteral">&quot;Orientation.jpg&quot;</span>, imgOrientation);</div></div><!-- fragment --><p> G-API-based functions like calcGST() can't be applied to input data directly, since it is a <em>construction</em> code, not the <em>processing</em> code. In order to <em>run</em> computations, a special object of class <a class="el" href="../../d9/dfe/classcv_1_1GComputation.html" title="GComputation class represents a captured computation graph. GComputation objects form boundaries for ...">cv::GComputation</a> needs to be created. This object wraps our G-API code (which is a composition of G-API data and operations) into a callable object, similar to C++11 <a href="https://en.cppreference.com/w/cpp/utility/functional/function">std::function&lt;&gt;</a>.</p>
<p><a class="el" href="../../d9/dfe/classcv_1_1GComputation.html" title="GComputation class represents a captured computation graph. GComputation objects form boundaries for ...">cv::GComputation</a> class has a number of constructors which can be used to define a graph. Generally, user needs to pass graph boundaries &ndash; <em>input</em> and <em>output</em> objects, on which a GComputation is defined. Then G-API analyzes the call flow from <em>outputs</em> to <em>inputs</em> and reconstructs the graph with operations in-between the specified boundaries. This may sound complex, however in fact the code looks like this:</p>
<div class="fragment"><div class="line">    <span class="comment">// Calculate Gradient Structure Tensor and post-process it for output with G-API</span></div><div class="line">    <a class="code" href="../../df/daa/classcv_1_1GMat.html">cv::GMat</a> in;</div><div class="line">    <a class="code" href="../../df/daa/classcv_1_1GMat.html">cv::GMat</a> imgCoherency, imgOrientation;</div><div class="line">    calcGST(in, imgCoherency, imgOrientation, W);</div><div class="line"></div><div class="line">    <a class="code" href="../../df/daa/classcv_1_1GMat.html">cv::GMat</a> imgCoherencyBin = imgCoherency &gt; C_Thr;</div><div class="line">    <a class="code" href="../../df/daa/classcv_1_1GMat.html">cv::GMat</a> imgOrientationBin = <a class="code" href="../../d0/d86/group__gapi__matrixop.html#ga76246465115220c5e5986a55298300c7">cv::gapi::inRange</a>(imgOrientation, LowThr, HighThr);</div><div class="line">    <a class="code" href="../../df/daa/classcv_1_1GMat.html">cv::GMat</a> imgBin = imgCoherencyBin &amp; imgOrientationBin;</div><div class="line">    <a class="code" href="../../df/daa/classcv_1_1GMat.html">cv::GMat</a> out = <a class="code" href="../../d0/d86/group__gapi__matrixop.html#ga60503b0500750881cc58a60e7eae084b">cv::gapi::addWeighted</a>(in, 0.5, imgBin, 0.5, 0.0);</div><div class="line"></div><div class="line">    <span class="comment">// Normalize extra outputs</span></div><div class="line">    <a class="code" href="../../df/daa/classcv_1_1GMat.html">cv::GMat</a> imgCoherencyNorm = <a class="code" href="../../d6/d91/group__gapi__transform.html#gaedda08934a12c8b4d8f3c1762f27cacc">cv::gapi::normalize</a>(imgCoherency, 0, 255, <a class="code" href="../../d2/de8/group__core__array.html#ggad12cefbcb5291cf958a85b4b67b6149fa9f0c1c342a18114d47b516a88e29822e">cv::NORM_MINMAX</a>);</div><div class="line">    <a class="code" href="../../df/daa/classcv_1_1GMat.html">cv::GMat</a> imgOrientationNorm = <a class="code" href="../../d6/d91/group__gapi__transform.html#gaedda08934a12c8b4d8f3c1762f27cacc">cv::gapi::normalize</a>(imgOrientation, 0, 255, <a class="code" href="../../d2/de8/group__core__array.html#ggad12cefbcb5291cf958a85b4b67b6149fa9f0c1c342a18114d47b516a88e29822e">cv::NORM_MINMAX</a>);</div><div class="line"></div><div class="line">    <span class="comment">// Capture the graph into object segm</span></div><div class="line">    <a class="code" href="../../d9/dfe/classcv_1_1GComputation.html">cv::GComputation</a> segm(<a class="code" href="../../d2/d75/namespacecv.html#a8e40d34081b18c79ba4c3cfb9fd0634f">cv::GIn</a>(in), <a class="code" href="../../d2/d75/namespacecv.html#adaa9a308669926cecc4793c6e4449629">cv::GOut</a>(out, imgCoherencyNorm, imgOrientationNorm));</div><div class="line"></div><div class="line">    <span class="comment">// Define cv::Mats for output data</span></div><div class="line">    <a class="code" href="../../d3/d63/classcv_1_1Mat.html">cv::Mat</a> imgOut, imgOutCoherency, imgOutOrientation;</div><div class="line"></div><div class="line">    <span class="comment">// Run the graph</span></div><div class="line">    segm.apply(<a class="code" href="../../d2/d75/namespacecv.html#a4941eb8cbe70b8c7187bc8459af7653b">cv::gin</a>(imgIn), <a class="code" href="../../d2/d75/namespacecv.html#ac82b8a261b82157293b603b55c096a9e">cv::gout</a>(imgOut, imgOutCoherency, imgOutOrientation));</div><div class="line"></div><div class="line">    <a class="code" href="../../d4/da8/group__imgcodecs.html#gabbc7ef1aa2edfaa87772f1202d67e0ce">cv::imwrite</a>(<span class="stringliteral">&quot;result.jpg&quot;</span>, imgOut);</div><div class="line">    <a class="code" href="../../d4/da8/group__imgcodecs.html#gabbc7ef1aa2edfaa87772f1202d67e0ce">cv::imwrite</a>(<span class="stringliteral">&quot;Coherency.jpg&quot;</span>, imgOutCoherency);</div><div class="line">    <a class="code" href="../../d4/da8/group__imgcodecs.html#gabbc7ef1aa2edfaa87772f1202d67e0ce">cv::imwrite</a>(<span class="stringliteral">&quot;Orientation.jpg&quot;</span>, imgOutOrientation);</div></div><!-- fragment --><p> Note that this code slightly changes from the original one: forming up the resulting image is also a part of the pipeline (done with <a class="el" href="../../d0/d86/group__gapi__matrixop.html#ga60503b0500750881cc58a60e7eae084b" title="Calculates the weighted sum of two matrices. ">cv::gapi::addWeighted</a>).</p>
<p>Result of this G-API pipeline bit-exact matches the original one (given the same input image):</p>
<div class="image">
<img src="../../result.jpg" alt="result.jpg"/>
<div class="caption">
Segmentation result with G-API</div></div>
 <h2><a class="anchor" id="gapi_anisotropic_ocv"></a>
G-API initial version: full listing</h2>
<p>Below is the full listing of the initial anisotropic image segmentation port on G-API:</p>
<div class="fragment"><div class="line"><span class="preprocessor">#include &lt;iostream&gt;</span></div><div class="line"><span class="preprocessor">#include &lt;utility&gt;</span></div><div class="line"></div><div class="line"><span class="preprocessor">#include &quot;<a class="code" href="../../d1/d4f/imgproc_2include_2opencv2_2imgproc_8hpp.html">opencv2/imgproc.hpp</a>&quot;</span></div><div class="line"><span class="preprocessor">#include &quot;<a class="code" href="../../d6/d87/imgcodecs_8hpp.html">opencv2/imgcodecs.hpp</a>&quot;</span></div><div class="line"><span class="preprocessor">#include &quot;<a class="code" href="../../d7/d20/gapi_8hpp.html">opencv2/gapi.hpp</a>&quot;</span></div><div class="line"><span class="preprocessor">#include &quot;<a class="code" href="../../d4/df9/gapi_2include_2opencv2_2gapi_2core_8hpp.html">opencv2/gapi/core.hpp</a>&quot;</span></div><div class="line"><span class="preprocessor">#include &quot;<a class="code" href="../../d4/d75/gapi_2include_2opencv2_2gapi_2imgproc_8hpp.html">opencv2/gapi/imgproc.hpp</a>&quot;</span></div><div class="line"></div><div class="line"><span class="keywordtype">void</span> calcGST(<span class="keyword">const</span> <a class="code" href="../../df/daa/classcv_1_1GMat.html">cv::GMat</a>&amp; inputImg, <a class="code" href="../../df/daa/classcv_1_1GMat.html">cv::GMat</a>&amp; imgCoherencyOut, <a class="code" href="../../df/daa/classcv_1_1GMat.html">cv::GMat</a>&amp; imgOrientationOut, <span class="keywordtype">int</span> w);</div><div class="line"></div><div class="line"><span class="keywordtype">int</span> main()</div><div class="line">{</div><div class="line">    <span class="keywordtype">int</span> W = 52;             <span class="comment">// window size is WxW</span></div><div class="line">    <span class="keywordtype">double</span> C_Thr = 0.43;    <span class="comment">// threshold for coherency</span></div><div class="line">    <span class="keywordtype">int</span> LowThr = 35;        <span class="comment">// threshold1 for orientation, it ranges from 0 to 180</span></div><div class="line">    <span class="keywordtype">int</span> HighThr = 57;       <span class="comment">// threshold2 for orientation, it ranges from 0 to 180</span></div><div class="line"></div><div class="line">    <a class="code" href="../../d3/d63/classcv_1_1Mat.html">cv::Mat</a> imgIn = <a class="code" href="../../d4/da8/group__imgcodecs.html#ga288b8b3da0892bd651fce07b3bbd3a56">cv::imread</a>(<span class="stringliteral">&quot;input.jpg&quot;</span>, <a class="code" href="../../d8/d6a/group__imgcodecs__flags.html#gga61d9b0126a3e57d9277ac48327799c80ae29981cfc153d3b0cef5c0daeedd2125">cv::IMREAD_GRAYSCALE</a>);</div><div class="line">    <span class="keywordflow">if</span> (imgIn.<a class="code" href="../../d3/d63/classcv_1_1Mat.html#abbec3525a852e77998aba034813fded4">empty</a>()) <span class="comment">//check whether the image is loaded or not</span></div><div class="line">    {</div><div class="line">        std::cout &lt;&lt; <span class="stringliteral">&quot;ERROR : Image cannot be loaded..!!&quot;</span> &lt;&lt; std::endl;</div><div class="line">        <span class="keywordflow">return</span> -1;</div><div class="line">    }</div><div class="line"></div><div class="line">    <span class="comment">// Calculate Gradient Structure Tensor and post-process it for output with G-API</span></div><div class="line">    <a class="code" href="../../df/daa/classcv_1_1GMat.html">cv::GMat</a> in;</div><div class="line">    <a class="code" href="../../df/daa/classcv_1_1GMat.html">cv::GMat</a> imgCoherency, imgOrientation;</div><div class="line">    calcGST(in, imgCoherency, imgOrientation, W);</div><div class="line"></div><div class="line">    <a class="code" href="../../df/daa/classcv_1_1GMat.html">cv::GMat</a> imgCoherencyBin = imgCoherency &gt; C_Thr;</div><div class="line">    <a class="code" href="../../df/daa/classcv_1_1GMat.html">cv::GMat</a> imgOrientationBin = <a class="code" href="../../d0/d86/group__gapi__matrixop.html#ga76246465115220c5e5986a55298300c7">cv::gapi::inRange</a>(imgOrientation, LowThr, HighThr);</div><div class="line">    <a class="code" href="../../df/daa/classcv_1_1GMat.html">cv::GMat</a> imgBin = imgCoherencyBin &amp; imgOrientationBin;</div><div class="line">    <a class="code" href="../../df/daa/classcv_1_1GMat.html">cv::GMat</a> out = <a class="code" href="../../d0/d86/group__gapi__matrixop.html#ga60503b0500750881cc58a60e7eae084b">cv::gapi::addWeighted</a>(in, 0.5, imgBin, 0.5, 0.0);</div><div class="line"></div><div class="line">    <span class="comment">// Normalize extra outputs</span></div><div class="line">    <a class="code" href="../../df/daa/classcv_1_1GMat.html">cv::GMat</a> imgCoherencyNorm = <a class="code" href="../../d6/d91/group__gapi__transform.html#gaedda08934a12c8b4d8f3c1762f27cacc">cv::gapi::normalize</a>(imgCoherency, 0, 255, <a class="code" href="../../d2/de8/group__core__array.html#ggad12cefbcb5291cf958a85b4b67b6149fa9f0c1c342a18114d47b516a88e29822e">cv::NORM_MINMAX</a>);</div><div class="line">    <a class="code" href="../../df/daa/classcv_1_1GMat.html">cv::GMat</a> imgOrientationNorm = <a class="code" href="../../d6/d91/group__gapi__transform.html#gaedda08934a12c8b4d8f3c1762f27cacc">cv::gapi::normalize</a>(imgOrientation, 0, 255, <a class="code" href="../../d2/de8/group__core__array.html#ggad12cefbcb5291cf958a85b4b67b6149fa9f0c1c342a18114d47b516a88e29822e">cv::NORM_MINMAX</a>);</div><div class="line"></div><div class="line">    <span class="comment">// Capture the graph into object segm</span></div><div class="line">    <a class="code" href="../../d9/dfe/classcv_1_1GComputation.html">cv::GComputation</a> segm(<a class="code" href="../../d2/d75/namespacecv.html#a8e40d34081b18c79ba4c3cfb9fd0634f">cv::GIn</a>(in), <a class="code" href="../../d2/d75/namespacecv.html#adaa9a308669926cecc4793c6e4449629">cv::GOut</a>(out, imgCoherencyNorm, imgOrientationNorm));</div><div class="line"></div><div class="line">    <span class="comment">// Define cv::Mats for output data</span></div><div class="line">    <a class="code" href="../../d3/d63/classcv_1_1Mat.html">cv::Mat</a> imgOut, imgOutCoherency, imgOutOrientation;</div><div class="line"></div><div class="line">    <span class="comment">// Run the graph</span></div><div class="line">    segm.apply(<a class="code" href="../../d2/d75/namespacecv.html#a4941eb8cbe70b8c7187bc8459af7653b">cv::gin</a>(imgIn), <a class="code" href="../../d2/d75/namespacecv.html#ac82b8a261b82157293b603b55c096a9e">cv::gout</a>(imgOut, imgOutCoherency, imgOutOrientation));</div><div class="line"></div><div class="line">    <a class="code" href="../../d4/da8/group__imgcodecs.html#gabbc7ef1aa2edfaa87772f1202d67e0ce">cv::imwrite</a>(<span class="stringliteral">&quot;result.jpg&quot;</span>, imgOut);</div><div class="line">    <a class="code" href="../../d4/da8/group__imgcodecs.html#gabbc7ef1aa2edfaa87772f1202d67e0ce">cv::imwrite</a>(<span class="stringliteral">&quot;Coherency.jpg&quot;</span>, imgOutCoherency);</div><div class="line">    <a class="code" href="../../d4/da8/group__imgcodecs.html#gabbc7ef1aa2edfaa87772f1202d67e0ce">cv::imwrite</a>(<span class="stringliteral">&quot;Orientation.jpg&quot;</span>, imgOutOrientation);</div><div class="line"></div><div class="line">    <span class="keywordflow">return</span> 0;</div><div class="line">}</div><div class="line"><span class="keywordtype">void</span> calcGST(<span class="keyword">const</span> <a class="code" href="../../df/daa/classcv_1_1GMat.html">cv::GMat</a>&amp; inputImg, <a class="code" href="../../df/daa/classcv_1_1GMat.html">cv::GMat</a>&amp; imgCoherencyOut, <a class="code" href="../../df/daa/classcv_1_1GMat.html">cv::GMat</a>&amp; imgOrientationOut, <span class="keywordtype">int</span> w)</div><div class="line">{</div><div class="line">    <span class="keyword">auto</span> img = <a class="code" href="../../d6/d91/group__gapi__transform.html#gae972b2c91904d36e92ee14a6778db82e">cv::gapi::convertTo</a>(inputImg, <a class="code" href="../../d1/d1b/group__core__hal__interface.html#ga4a3def5d72b74bed31f5f8ab7676099c">CV_32F</a>);</div><div class="line">    <span class="keyword">auto</span> imgDiffX = <a class="code" href="../../da/dc5/group__gapi__filters.html#gae1443445c4cb3187dcf439a57cfa534f">cv::gapi::Sobel</a>(img, <a class="code" href="../../d1/d1b/group__core__hal__interface.html#ga4a3def5d72b74bed31f5f8ab7676099c">CV_32F</a>, 1, 0, 3);</div><div class="line">    <span class="keyword">auto</span> imgDiffY = <a class="code" href="../../da/dc5/group__gapi__filters.html#gae1443445c4cb3187dcf439a57cfa534f">cv::gapi::Sobel</a>(img, <a class="code" href="../../d1/d1b/group__core__hal__interface.html#ga4a3def5d72b74bed31f5f8ab7676099c">CV_32F</a>, 0, 1, 3);</div><div class="line">    <span class="keyword">auto</span> imgDiffXY = <a class="code" href="../../da/dd3/group__gapi__math.html#ga42a8187b2bf350f90e4a31f93e929e51">cv::gapi::mul</a>(imgDiffX, imgDiffY);</div><div class="line"></div><div class="line">    <span class="keyword">auto</span> imgDiffXX = <a class="code" href="../../da/dd3/group__gapi__math.html#ga42a8187b2bf350f90e4a31f93e929e51">cv::gapi::mul</a>(imgDiffX, imgDiffX);</div><div class="line">    <span class="keyword">auto</span> imgDiffYY = <a class="code" href="../../da/dd3/group__gapi__math.html#ga42a8187b2bf350f90e4a31f93e929e51">cv::gapi::mul</a>(imgDiffY, imgDiffY);</div><div class="line"></div><div class="line">    <span class="keyword">auto</span> J11 = <a class="code" href="../../da/dc5/group__gapi__filters.html#gaa37a58b30753913cbcc7f5425726dbd5">cv::gapi::boxFilter</a>(imgDiffXX, <a class="code" href="../../d1/d1b/group__core__hal__interface.html#ga4a3def5d72b74bed31f5f8ab7676099c">CV_32F</a>, <a class="code" href="../../d6/d50/classcv_1_1Size__.html">cv::Size</a>(w, w));</div><div class="line">    <span class="keyword">auto</span> J22 = <a class="code" href="../../da/dc5/group__gapi__filters.html#gaa37a58b30753913cbcc7f5425726dbd5">cv::gapi::boxFilter</a>(imgDiffYY, <a class="code" href="../../d1/d1b/group__core__hal__interface.html#ga4a3def5d72b74bed31f5f8ab7676099c">CV_32F</a>, <a class="code" href="../../d6/d50/classcv_1_1Size__.html">cv::Size</a>(w, w));</div><div class="line">    <span class="keyword">auto</span> J12 = <a class="code" href="../../da/dc5/group__gapi__filters.html#gaa37a58b30753913cbcc7f5425726dbd5">cv::gapi::boxFilter</a>(imgDiffXY, <a class="code" href="../../d1/d1b/group__core__hal__interface.html#ga4a3def5d72b74bed31f5f8ab7676099c">CV_32F</a>, <a class="code" href="../../d6/d50/classcv_1_1Size__.html">cv::Size</a>(w, w));</div><div class="line"></div><div class="line">    <span class="keyword">auto</span> tmp1 = J11 + J22;</div><div class="line">    <span class="keyword">auto</span> tmp2 = J11 - J22;</div><div class="line">    <span class="keyword">auto</span> tmp22 = <a class="code" href="../../da/dd3/group__gapi__math.html#ga42a8187b2bf350f90e4a31f93e929e51">cv::gapi::mul</a>(tmp2, tmp2);</div><div class="line">    <span class="keyword">auto</span> tmp3 = <a class="code" href="../../da/dd3/group__gapi__math.html#ga42a8187b2bf350f90e4a31f93e929e51">cv::gapi::mul</a>(J12, J12);</div><div class="line">    <span class="keyword">auto</span> tmp4 = <a class="code" href="../../da/dd3/group__gapi__math.html#gaf3880c8feae9027fa91a701a1fd0d3b4">cv::gapi::sqrt</a>(tmp22 + 4.0*tmp3);</div><div class="line"></div><div class="line">    <span class="keyword">auto</span> lambda1 = tmp1 + tmp4;</div><div class="line">    <span class="keyword">auto</span> lambda2 = tmp1 - tmp4;</div><div class="line"></div><div class="line">    imgCoherencyOut = (lambda1 - lambda2) / (lambda1 + lambda2);</div><div class="line">    imgOrientationOut = 0.5*<a class="code" href="../../da/dd3/group__gapi__math.html#ga3c98a8ad1054d296500220737c04d18c">cv::gapi::phase</a>(J22 - J11, 2.0*J12, <span class="keyword">true</span>);</div><div class="line">}</div><div class="line"></div></div><!-- fragment --> <h1><a class="anchor" id="gapi_anisotropic_inspect"></a>
Inspecting the initial version</h1>
<p>After we have got the initial working version of our algorithm working with G-API, we can use it to inspect and learn how G-API works. This chapter covers two aspects: understanding the graph structure, and memory profiling.</p>
<h2><a class="anchor" id="gapi_anisotropic_inspect_graph"></a>
Understanding the graph structure</h2>
<p>G-API stands for "Graph API", but did you mention any graphs in the above example? It was one of the initial design goals &ndash; G-API was designed with expressions in mind to make adoption and porting process more straightforward. People <em>usually</em> don't think in terms of <em>Nodes</em> and <em>Edges</em> when writing ordinary code, and so G-API, while being a Graph API, doesn't force its users to do that.</p>
<p>However, a graph is still built implicitly when a <a class="el" href="../../d9/dfe/classcv_1_1GComputation.html" title="GComputation class represents a captured computation graph. GComputation objects form boundaries for ...">cv::GComputation</a> object is defined. It may be useful to inspect how the resulting graph looks like to check if it is generated correctly and if it really represents our alrogithm. It is also useful to learn the structure of the graph to see if it has any redundancies.</p>
<p>G-API allows to dump generated graphs to <code>.dot</code> files which then could be visualized with <a href="https://www.graphviz.org/">Graphviz</a>, a popular open graph visualization software.</p>
<p>In order to dump our graph to a <code>.dot</code> file, set <code>GRAPH_DUMP_PATH</code> to a file name before running the application, e.g.: </p><pre class="fragment">$ GRAPH_DUMP_PATH=segm.dot ./bin/example_tutorial_porting_anisotropic_image_segmentation_gapi
</pre><p>Now this file can be visualized with a <code>dot</code> command like this: </p><pre class="fragment">$ dot segm.dot -Tpng -o segm.png
</pre><p>or viewed interactively with <code>xdot</code> (please refer to your distribution/operating system documentation on how to install these packages).</p>
<div class="image">
<img src="../../segm.gif" alt="segm.gif"/>
<div class="caption">
Anisotropic image segmentation graph</div></div>
<p> The above diagram demonstrates a number of interesting aspects of G-API's internal algorithm representation:</p><ol type="1">
<li>G-API underlying graph is a bipartite graph: it consists of <em>Operation</em> and <em>Data</em> nodes such that a <em>Data</em> node can only be connected to an <em>Operation</em> node, <em>Operation</em> node can only be connected to a <em>Data</em> node, and nodes of a single kind are never connected directly.</li>
<li>Graph is directed - every edge in the graph has a direction.</li>
<li>Graph "begins" and "ends" with a <em>Data</em> kind of nodes.</li>
<li>A <em>Data</em> node can have only a single writer and multiple readers.</li>
<li>An <em>Operation</em> node may have multiple inputs, though every input must have an unique <em>port number</em> (among inputs).</li>
<li>An <em>Operation</em> node may have multiple outputs, and every output must have an unique <em>port number</em> (among outputs).</li>
</ol>
<h2><a class="anchor" id="gapi_anisotropic_memory_ocv"></a>
Measuring memory footprint</h2>
<p>Let's measure and compare memory footprint of the algorithm in its two versions: G-API-based and OpenCV-based. At the moment, G-API version is also OpenCV-based since it fallbacks to OpenCV functions inside.</p>
<p>On GNU/Linux, application memory footprint can be profiled with <a href="http://valgrind.org/">Valgrind</a>. On Debian/Ubuntu systems it can be installed like this (assuming you have administrator privileges): </p><pre class="fragment">$ sudo apt-get install valgrind massif-visualizer
</pre><p>Once installed, we can collect memory profiles easily for our two algorithm versions: </p><pre class="fragment">$ valgrind --tool=massif --massif-out-file=ocv.out ./bin/example_tutorial_anisotropic_image_segmentation
==6101== Massif, a heap profiler
==6101== Copyright (C) 2003-2015, and GNU GPL'd, by Nicholas Nethercote
==6101== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info
==6101== Command: ./bin/example_tutorial_anisotropic_image_segmentation
==6101==
==6101==
$ valgrind --tool=massif --massif-out-file=gapi.out ./bin/example_tutorial_porting_anisotropic_image_segmentation_gapi
==6117== Massif, a heap profiler
==6117== Copyright (C) 2003-2015, and GNU GPL'd, by Nicholas Nethercote
==6117== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info
==6117== Command: ./bin/example_tutorial_porting_anisotropic_image_segmentation_gapi
==6117==
==6117==
</pre><p>Once done, we can inspect the collected profiles with <a href="https://github.com/KDE/massif-visualizer">Massif Visualizer</a> (installed in the above step).</p>
<p>Below is the visualized memory profile of the original OpenCV version of the algorithm:</p>
<div class="image">
<img src="../../massif_export_ocv.png" alt="massif_export_ocv.png"/>
<div class="caption">
Memory profile: original Anisotropic Image Segmentation sample</div></div>
<p> We see that memory is allocated as the application executes, reaching its peak in the calcGST() function; then the footprint drops as calcGST() completes its execution and all temporary buffers are freed. Massif reports us peak memory consumption of 7.6 MiB.</p>
<p>Now let's have a look on the profile of G-API version:</p>
<div class="image">
<img src="../../massif_export_gapi.png" alt="massif_export_gapi.png"/>
<div class="caption">
Memory profile: G-API port of Anisotropic Image Segmentation sample</div></div>
<p> Once G-API computation is created and its execution starts, G-API allocates all required memory at once and then the memory profile remains flat until the termination of the program. Massif reports us peak memory consumption of 11.4 MiB.</p>
<p>A reader may ask a right question at this point &ndash; is G-API that bad? What is the reason in using it than?</p>
<p>Hopefully, it is not. The reason why we see here an increased memory consumption is because the default naive OpenCV-based backend is used to execute this graph. This backend serves mostly for quick prototyping and debugging algorithms before offload/further optimization.</p>
<p>This backend doesn't utilize any complex memory management strategies yet since it is not its point at the moment. In the following chapter, we'll learn about Fluid backend and see how the same G-API code can run in a completely different model (and the footprint shrunk to a number of kilobytes).</p>
<h1><a class="anchor" id="gapi_anisotropic_backends"></a>
Backends and kernels</h1>
<p>This chapter covers how a G-API computation can be executed in a special way &ndash; e.g. offloaded to another device, or scheduled with a special intelligence. G-API is designed to make its graphs portable &ndash; it means that once a graph is defined in G-API terms, no changes should be required in it if we want to run it on CPU or on GPU or on both devices at once. <a class="el" href="../../de/d4d/gapi_hld.html">G-API High-level overview</a> and <a class="el" href="../../d0/d25/gapi_kernel_api.html">G-API Kernel API</a> shed more light on technical details which make it possible. In this chapter, we will utilize G-API Fluid backend to make our graph cache-efficient on CPU.</p>
<p>G-API defines <em>backend</em> as the lower-level entity which knows how to run kernels. Backends may have (and, in fact, do have) different <em>Kernel APIs</em> which are used to program and integrate kernels for that backends. In this context, <em>kernel</em> is an implementation of an <em>operation</em>, which is defined on the top API level (see <a class="el" href="../../de/d7f/gkernel_8hpp.html#a83d2e73c619a545f69e89bf7b03cbd80">G_TYPED_KERNEL()</a> macro).</p>
<p>Backend is a thing which is aware of device &amp; platform specifics, and which executes its kernels with keeping that specifics in mind. For example, there may be <a href="http://halide-lang.org/">Halide</a> backend which allows to write (implement) G-API operations in Halide language and then generate functional Halide code for portions of G-API graph which map well there.</p>
<h2><a class="anchor" id="gapi_anisotropic_fluid"></a>
Running a graph with a Fluid backend</h2>
<p>OpenCV 4.0 is bundled with two G-API backends &ndash; the default "OpenCV" which we just used, and a special "Fluid" backend.</p>
<p>Fluid backend reorganizes the execution to save memory and to achieve near-perfect cache locality, implementing so-called "streaming" model of execution.</p>
<p>In order to start using Fluid kernels, we need first to include appropriate header files (which are not included by default):</p>
<div class="fragment"><div class="line"><span class="preprocessor">#include &quot;<a class="code" href="../../d3/d81/gapi_2include_2opencv2_2gapi_2fluid_2core_8hpp.html">opencv2/gapi/fluid/core.hpp</a>&quot;</span>            <span class="comment">// Fluid Core kernel library</span></div><div class="line"><span class="preprocessor">#include &quot;<a class="code" href="../../de/da0/gapi_2include_2opencv2_2gapi_2fluid_2imgproc_8hpp.html">opencv2/gapi/fluid/imgproc.hpp</a>&quot;</span>         <span class="comment">// Fluid ImgProc kernel library</span></div></div><!-- fragment --><p> Once these headers are included, we can form up a new <em>kernel package</em> and specify it to G-API:</p>
<div class="fragment"><div class="line">    <span class="comment">// Prepare the kernel package and run the graph</span></div><div class="line">    <a class="code" href="../../de/db6/classcv_1_1gapi_1_1GKernelPackage.html">cv::gapi::GKernelPackage</a> fluid_kernels = <a class="code" href="../../d4/d1c/namespacecv_1_1gapi.html#ab3c55a390c722279ced6f56523fa01a7">cv::gapi::combine</a>        <span class="comment">// Define a custom kernel package:</span></div><div class="line">        (<a class="code" href="../../d9/d29/group__gapi__compile__args.html#ga18c46d5801429bb63848fc6e2391cb20">cv::gapi::core::fluid::kernels</a>(),                            <span class="comment">// ...with Fluid Core kernels</span></div><div class="line">         <a class="code" href="../../d9/d29/group__gapi__compile__args.html#ga18c46d5801429bb63848fc6e2391cb20">cv::gapi::imgproc::fluid::kernels</a>());                        <span class="comment">// ...and Fluid ImgProc kernels</span></div></div><!-- fragment --><p> In G-API, kernels (or operation implementations) are objects. Kernels are organized into collections, or <em>kernel packages</em>, represented by class <a class="el" href="../../de/db6/classcv_1_1gapi_1_1GKernelPackage.html" title="A container class for heterogeneous kernel implementation collections and graph transformations. ">cv::gapi::GKernelPackage</a>. The main purpose of a kernel package is to capture which kernels we would like to use in our graph, and pass it as a <em>graph compilation option</em>:</p>
<div class="fragment"><div class="line">    segm.apply(<a class="code" href="../../d2/d75/namespacecv.html#a4941eb8cbe70b8c7187bc8459af7653b">cv::gin</a>(imgIn),                                        <span class="comment">// Input data vector</span></div><div class="line">               <a class="code" href="../../d2/d75/namespacecv.html#ac82b8a261b82157293b603b55c096a9e">cv::gout</a>(imgOut, imgOutCoherency, imgOutOrientation),  <span class="comment">// Output data vector</span></div><div class="line">               <a class="code" href="../../d9/d29/group__gapi__compile__args.html#ga3ccf2a52953f18bb3e4c01243cc4e679">cv::compile_args</a>(fluid_kernels));                      <span class="comment">// Kernel package to use</span></div></div><!-- fragment --><p> Traditional OpenCV is logically divided into modules, with every module providing a set of functions. In G-API, there are also "modules" which are represented as kernel packages provided by a particular backend. In this example, we pass Fluid kernel packages to G-API to utilize appropriate Fluid functions in our graph.</p>
<p>Kernel packages are combinable &ndash; in the above example, we take "Core" and "ImgProc" Fluid kernel packages and combine it into a single one. See documentation reference on <a class="el" href="../../d4/d1c/namespacecv_1_1gapi.html#ab3c55a390c722279ced6f56523fa01a7">cv::gapi::combine</a>.</p>
<p>If no kernel packages are specified in options, G-API is using <em>default</em> package which consists of default OpenCV implementations and thus G-API graphs are executed via OpenCV functions by default. OpenCV backend provides broader functional coverage than any other backend. If a kernel package is specified, like in this example, then it is being combined with the <em>default</em>. It means that user-specified implementations will replace default implementations in case of conflict.</p>
<h2><a class="anchor" id="gapi_anisotropic_trouble"></a>
Troubleshooting and customization</h2>
<p>After the above modifications, (in OpenCV 4.0) the app should crash with a message like this:</p>
<div class="fragment"><div class="line">$ ./bin/example_tutorial_porting_anisotropic_image_segmentation_gapi_fluid</div><div class="line">terminate called after throwing an instance of &#39;std::logic_error&#39;</div><div class="line">  what():  .../modules/gapi/src/backends/fluid/gfluidimgproc.cpp:436: Assertion kernelSize.width == 3 &amp;&amp; kernelSize.height == 3 in function run failed</div><div class="line"></div><div class="line">Aborted (core dumped)</div></div><!-- fragment --><p>Fluid backend has a number of limitations in OpenCV 4.0 (see this <a href="https://github.com/opencv/opencv/wiki/Graph-API">wiki page</a> for a more up-to-date status). In particular, the Box filter used in this sample supports only static 3x3 kernel size.</p>
<p>We can overcome this problem easily by avoiding G-API using Fluid version of Box filter kernel in this sample. It can be done by removing the appropriate kernel from the kernel package we've just created:</p>
<div class="fragment"><div class="line">    fluid_kernels.<a class="code" href="../../de/db6/classcv_1_1gapi_1_1GKernelPackage.html#aef7392cdc72aafbd304abf42b15f98fb">remove</a>&lt;cv::gapi::imgproc::GBoxFilter&gt;();            <span class="comment">// Remove Fluid Box filter as unsuitable,</span></div><div class="line">                                                                      <span class="comment">// G-API will fall-back to OpenCV there.</span></div></div><!-- fragment --><p> Now this kernel package doesn't have <em>any</em> implementation of Box filter kernel interface (specified as a template parameter). As described above, G-API will fall-back to OpenCV to run this kernel now. The resulting code with this change now looks like:</p>
<div class="fragment"><div class="line">    <span class="comment">// Prepare the kernel package and run the graph</span></div><div class="line">    <a class="code" href="../../de/db6/classcv_1_1gapi_1_1GKernelPackage.html">cv::gapi::GKernelPackage</a> fluid_kernels = <a class="code" href="../../d4/d1c/namespacecv_1_1gapi.html#ab3c55a390c722279ced6f56523fa01a7">cv::gapi::combine</a>        <span class="comment">// Define a custom kernel package:</span></div><div class="line">        (<a class="code" href="../../d9/d29/group__gapi__compile__args.html#ga18c46d5801429bb63848fc6e2391cb20">cv::gapi::core::fluid::kernels</a>(),                            <span class="comment">// ...with Fluid Core kernels</span></div><div class="line">         <a class="code" href="../../d9/d29/group__gapi__compile__args.html#ga18c46d5801429bb63848fc6e2391cb20">cv::gapi::imgproc::fluid::kernels</a>());                        <span class="comment">// ...and Fluid ImgProc kernels</span></div><div class="line"><span class="comment"></span>    fluid_kernels.<a class="code" href="../../de/db6/classcv_1_1gapi_1_1GKernelPackage.html#aef7392cdc72aafbd304abf42b15f98fb">remove</a>&lt;cv::gapi::imgproc::GBoxFilter&gt;();            <span class="comment">// Remove Fluid Box filter as unsuitable,</span></div><div class="line">                                                                      <span class="comment">// G-API will fall-back to OpenCV there.</span></div><div class="line"><span class="comment"></span>    segm.apply(<a class="code" href="../../d2/d75/namespacecv.html#a4941eb8cbe70b8c7187bc8459af7653b">cv::gin</a>(imgIn),                                        <span class="comment">// Input data vector</span></div><div class="line">               <a class="code" href="../../d2/d75/namespacecv.html#ac82b8a261b82157293b603b55c096a9e">cv::gout</a>(imgOut, imgOutCoherency, imgOutOrientation),  <span class="comment">// Output data vector</span></div><div class="line">               <a class="code" href="../../d9/d29/group__gapi__compile__args.html#ga3ccf2a52953f18bb3e4c01243cc4e679">cv::compile_args</a>(fluid_kernels));                      <span class="comment">// Kernel package to use</span></div></div><!-- fragment --><p> Let's examine the memory profile for this sample after we switched to Fluid backend. Now it looks like this:</p>
<div class="image">
<img src="../../massif_export_gapi_fluid.png" alt="massif_export_gapi_fluid.png"/>
<div class="caption">
Memory profile: G-API/Fluid port of Anisotropic Image Segmentation sample</div></div>
<p> Now the tool reports 4.7MiB &ndash; and we just changed a few lines in our code, without modifying the graph itself! It is a ~2.4X improvement of the previous G-API result, and ~1.6X improvement of the original OpenCV version.</p>
<p>Let's also examine how the internal representation of the graph now looks like. Dumping the graph into <code>.dot</code> would result into a visualization like this:</p>
<div class="image">
<img src="../../segm_fluid.gif" alt="segm_fluid.gif"/>
<div class="caption">
Anisotropic image segmentation graph with OpenCV &amp; Fluid kernels</div></div>
<p> This graph doesn't differ structurally from its previous version (in terms of operations and data objects), though a changed layout (on the left side of the dump) is easily noticeable.</p>
<p>The visualization reflects how G-API deals with mixed graphs, also called <em>heterogeneous</em> graphs. The majority of operations in this graph are implemented with Fluid backend, but Box filters are executed by the OpenCV backend. One can easily see that the graph is partitioned (with rectangles). G-API groups connected operations based on their affinity, forming <em>subgraphs</em> (or <em>islands</em> in G-API terminology), and our top-level graph becomes a composition of multiple smaller subgraphs. Every backend determines how its subgraph (island) is executed, so Fluid backend optimizes out memory where possible, and six intermediate buffers accessed by OpenCV Box filters are allocated fully and can't be optimized out.</p>
<h1><a class="anchor" id="gapi_tutor_conclusion"></a>
Conclusion</h1>
<p>This tutorial demonstrates what G-API is and what its key design concepts are, how an algorithm can be ported to G-API, and how to utilize graph model benefits after that.</p>
<p>In OpenCV 4.0, G-API is still in its inception stage &ndash; it is more a foundation for all future work, though ready for use even now.</p>
<p>Further, this tutorial will be extended with new chapters on custom kernels programming, parallelism, and more. </p>
</div></div><!-- contents -->
<!-- HTML footer for doxygen 1.8.6-->
<!-- start footer part -->
<hr class="footer"/><address class="footer"><small>
Generated on Fri Apr 2 2021 11:36:34 for OpenCV by &#160;<a href="http://www.doxygen.org/index.html">
<img class="footer" src="../../doxygen.png" alt="doxygen"/>
</a> 1.8.13
</small></address>
<script type="text/javascript">
//<![CDATA[
addTutorialsButtons();
//]]>
</script>
</body>
</html>
